System and Method for Knowledge Verification Utilizing Biopotentials and Physiologic Metrics

ABSTRACT

A system and method for knowledge verification utilizing biopotentials and physiologic metrics, which includes a computer-based device having stored thereon Probe, Relevant and Gallery image data, and a biopotential amplifier removably connected to a human subject via disposable Ag/Ag—Cl electrodes. Furthermore, the system comprises an analog-to-digital (A/D) converter to digitize said biopotential data for subsequent storage on said computer-based device, analysis software for discriminating said subject&#39;s event-related response to the exogenous stimuli, a visual display system comprising an LCD video monitor, and control software for presenting the Probe, Relevant and Gallery visual stimuli in a weighted, pseudo-random sequence which can be modulated by the outcome of said analysis software. Probe image data are not generally known to said human subjects but relevant to the knowledge to be verified; Relevant image data are generally known to said human subjects but not relevant to the knowledge to be verified; and Gallery image data are not generally known to said human subjects and not relevant to the knowledge to be verified. Said knowledge verification system can utilize parametric or non-parametric, e.g., artificial neural networks, analysis to provide an output of verification, or non-verification of knowledge of interest. Exemplary headband and electrode configurations optimized to produce the desired signals are disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.12/389,722, filed 20 Feb. 2009, now pending, which claims the benefit ofU.S. Provisional Patent Application No. 61/066,750, filed 25 Feb. 2008.

BACKGROUND

1. Field

The use of physiological methods to detect concealed information is asold as civilization. Even in ancient times observers noticed thatwhenever a crime suspect was being interrogated, the fear of possibleexposure caused certain changes in his physiological functions. Inancient China a suspect was required to hold a quantity of rice in hismouth throughout the reading of his verdict. If the rice was still dryat the end, then the suspects guilt was considered proven since it wasbelieved the fear of being exposed significantly reduced salivation inhis mouth. Related techniques have been used throughout history by manyvery different peoples all over there world. However, modern“instrument-aided” physiological detection techniques did not ariseuntil the early 20^(th) century.

2. Prior Art

William Marston, an American psychologist and attorney, stands apartfrom the rest of the early research community studying instrument-aided“lie detection.” In 1913 the US National Research Committee formed agroup of psychologists, including William Marston, to assess thepotential use of known methods of lie detection for the needs of thecounterintelligence service during World War I. Having conducted asubstantial amount of research, the group concluded that the “bloodpressure deception test” developed by Marston at Harvard University'spsychology lab around 1913 was the most accurate method of all known atthat time. Its accuracy in detecting lies was estimated to be as high as97 percent. In 1923, John Marston, introduced for the first time,polygraph examination findings as proof of evidence in a court of law.

John Larson, a California police officer, designed the first practicalprototype of a modern polygraph. After learning about Marston's “bloodpressure deception test”, Larson developed an instrument capable ofcontinuously recording blood pressure, pulse and respiration. Larsonthen set about developing an interviewing protocol, which was called theRelevant/Irrelevant (R/I) procedure. Throughout the interrogation, hewould sprinkle questions relevant to the crime (“Do you own a .38revolver?”) and questions that had nothing to do with it (“Are youtwenty-eight years old?”). The assumption he made was that the innocentwould have a similar physiological response to both types of questions,but guilty people would react more strongly to crime-relevant questions.The key problem with this approach was that even innocent people mightbe nervous, and crime-specific questions are generally fairly obvious.Nonetheless, a large number of criminal suspects were examined usingthis equipment, and highly accurate results were achieved.

Leonarde Keeler, working under Larson's guidance during the 1920s,played a crucial role in the deployment of the psycho-physiologicalmethod of lie detection in the United States. He came up with the firstpolygraph specifically designed to elicit hidden information (1933), thefirst guidebook on examination using a lie detector (1935), and foundeda company for commercial production of these instruments. He alsoestablished the first training facility for polygraph examiners. Lastly,Mr. Keeler was the first to introduce polygraphy in the area ofrecruitment and crime prevention in business.

By the end of the 1930s three US companies were mass-producing liedetectors and marketing them throughout North America. Almost onehundred police departments in 28 states were making ample use of thisinstrument in their everyday work, and dozens of banks and commercialfirms in the northern states introduced polygraphy for recruitingpurposes and in-house investigations.

At about the time of the start of World War II the AmericanPsychological Society undertook a special study to verify thereliability of polygraph examination in the interest of protecting thepublic. Having thoroughly analyzed the use of polygraph examinations inlaw-enforcement and the business environment, the research committeeconcluded that lie detection methods were sufficiently developed; thenecessary technology existed; and a good number of well-trainedspecialists were available. Of these three factors, the last wasconsidered most important. It was agreed that if a competent specialistwas available, the examination results were quite useful. If suchspecialists were not available, it was concluded the method nor theequipment should be used.

At this point confidence in polygraphs started to increase andconsequently their popularity did as well. Beginning in the early 1940sthe polygraph was extensively used in protecting state secrets.Polygraphs were used in checking the personnel that had worked on thenuclear bomb project at the Oak Ridge Research Center.

Modern polygraphy consists of a computer system with bio-sensors.Sensors are used to measure and record a number of physiological changesthat are related to the involuntary nervous system. The reliability ofpolygraphy is directly related to the number of measured and recordedinputs: typically the more inputs used, the more accurate the result.Decision-making is made by a trained expert human operator and is basedon the aggregate of measurements taken, as well as individualcharacteristics.

Experts know that there is no direct connection between physicalindicators and the sincerity of a person. Lie detectors record the levelof involuntary nervous behavior of the examinee, but fail to identifythe causative agent for changes measured by the instrument. Thepolygraph examiner has to make the final determination based onsubjective input.

Functional Magnetic Resonance Imaging and Truth Detection

It has been known for over 100 years that changes in blood flow andblood oxygenation in the brain (collectively known as hemodynamics) areclosely linked to neural activity. When nerve cells are active theyconsume oxygen carried by hemoglobin in red blood cells from localcapillaries. The local response to this oxygen utilization is anincrease in blood flow to regions of increased neural activity,occurring after a delay of approximately 1-5 seconds. This hemodynamicresponse rises to a peak over 4-5 seconds, before falling back tobaseline (and typically undershooting slightly). This leads to localchanges in the relative concentration of oxyhemoglobin anddeoxyhemoglobin and changes in local cerebral blood volume in additionto change in local cerebral blood flow.

Hemoglobin is diamagnetic when oxygenated but paramagnetic whendeoxygenated. The magnetic resonance (MR) signal of blood is thereforeslightly different depending on the level of oxygenation. Thesedifferential signals can be detected using an appropriate MR pulsesequence, and manifest themselves as a blood-oxygen-level dependent(BOLD) contrast. Higher BOLD signal intensities arise from decreases inthe concentration of deoxygenated hemoglobin since the blood magneticsusceptibility now more closely matches the tissue magneticsusceptibility. By collecting data in an MRI scanner with parameterssensitive to changes in magnetic susceptibility, one can assess changesin BOLD contrast. These changes can be either positive or negativedepending upon the relative changes in both cerebral blood flow (CBF)and oxygen consumption. Increases in CBF that outstrip changes in oxygenconsumption will lead to an increased BOLD signal. Conversely decreasesin CBF that outstrip changes in oxygen consumption will cause adecreased BOLD intensity.

The precise relationship between neural signals and BOLD is still underactive research. In general, changes in the BOLD signal are wellcorrelated with changes in blood flow. Numerous studies during the pastseveral decades have identified a coupling between blood flow andmetabolic rate; that is, the blood supply is tightly regulated in spaceand time to provide the nutrients for brain metabolism. However,neuroscientists have been seeking a more direct relationship between theblood supply and the neural inputs/outputs that can be related toobservable electrical activity and circuit models of brain function.

Although current data indicate that local field potentials, an index ofintegrated electrical activity, form a better correlation with bloodflow than the spiking action potentials that are most directlyassociated with neural communication, no simple measure of electricalactivity to date has provided an adequate correlation with metabolismand the blood supply across a wide dynamic range. Presumably, thisreflects the complex nature of metabolic processes, which form asuperset with regards to electrical activity. Some recent results havesuggested that the increase in CBF following neural activity is notcausally related to the metabolic demands of the brain region, butrather is driven by the presence of neurotransmitters, especiallyglutamate.

Some other recent results suggest that an initial small, negative dipbefore the main positive BOLD signal is more highly localized and alsocorrelates with measured local decreases in tissue oxygen concentration(perhaps reflecting increased local metabolism during neuronactivation). Use of this more localized negative BOLD signal has enabledimaging of human ocular dominance columns in primary visual cortex, withresolution of about 0.5 mm. One problem with this technique is that theearly negative BOLD signal is small and can only be seen using largerscanners with magnetic fields of at least 3 Teslas. Further, the signalis much smaller than the normal BOLD signal, making extraction of thesignal from noise that much more difficult. Also, this initial dipoccurs within 1-2 seconds of stimulus initiation, which may not becaptured when signals are recorded at long repetition (TR). If the TR issufficiently low, increased speed of the cerebral blood flow responsedue to consumption of vasoactive drugs (such as caffeine) or naturaldifferences in vascular responsiveness may further obscure observationof the initial dip.

The BOLD signal is composed of CBF contributions from larger arteriesand veins, smaller arterioles and venules, and capillaries. Experimentalresults indicate that the BOLD signal can be weighted to the smallervessels, and hence closer to the active neurons, by using largermagnetic fields. For example, whereas about 70% of the BOLD signalarises from larger vessels in a 1.5 Tesla scanner, about 70% arises fromsmaller vessels in a 4 Tesla scanner. Furthermore, the size of the BOLDsignal increases roughly as the square of the magnetic field strength.Hence there has been a push for larger field scanners to both improvelocalization and increase the signal. A few 7 Tesla commercial scannershave become operational, and experimental 8 and 9 Tesla scanners areunder development.

BOLD effects are measured using rapid volumetric acquisition of images.Such images can be acquired with moderately good spatial and temporalresolution; images are usually taken every 1-4 seconds, and the sectionsin the resulting image typically represent cubes of tissue about 2-4millimeters on each side in humans. Recent technical advancements, suchas the use of high magnetic fields and advanced “multi-channel” RFreception, have advanced spatial resolution to the millimeter scale.Although responses to stimuli presented as close together as one or twoseconds can be distinguished from one another, using a method known asevent-related functional Magnetic Resonance Imaging (fMRI), the fulltime course of a BOLD response to a briefly presented stimulus lastsabout 15 seconds for the robust positive response.

The science behind fMRI lie detection has matured with astonishingspeed. One of the pioneers in the field is Daniel Langleben, apsychiatrist at the University of Pennsylvania. Langleben developed ahypothesis that in order to formulate a lie, the brain first had to stopitself from telling the truth and then generate the deception—a processthat could be mapped with fMRI. By analyzing time-sequenced BOLD signalsources, fMRI reveals the pathways that thoughts have taken through thebrain. Langleben concluded in 2002 in a paper published in the journalNeuroImage that there is “a neurophysiological difference betweendeception and truth” that can be detected with fMRI.

The subject took on a new urgency after 9/11 as security shot to the topof the national agenda. Despite questions about reliability, the use ofpolygraph machines grew rapidly, both domestically—where the device isemployed to evaluate government workers for security clearances—and inplaces like Iraq and Afghanistan, where polygraphers are deployed toextract confessions, check claims about weapons of mass destruction,confirm the loyalty of coalition officers, and grill spies. TheDepartment of Defense Polygraph Institute (DoDPI) put out a call forfunding requests to scientists investigating lie detection. Grants fromDoDPI, the Department of Homeland Security, DARPA, and other agenciestriggered a wave of research into new lie-detection technologies. Sincethe events of 9/11, there are now over 50 labs in the US alone doingthis kind of research.

Langleben's team, whose work was funded partially by DARPA, began tofocus on eliminating one major source of polygraph error—thesubjectivity of the human examiner. Langleben and his colleaguesdeveloped pattern-recognition algorithms that identify deception inindividual subjects by comparing their brain scans with those in adatabase of known liars. In 2005, both Langleben's lab and aDoDPI-funded team led by Andrew Kozel at the Medical University of SouthCarolina announced that their algorithms had been able to reliablyidentify lies.

Today's fMRI scanners are bulky, cost up to $3 million each, and ineffect require consent because of their sensitivity to head movement.This technology is not considered applicable to individuals who mightwant to keep information concealed, and in spite of the advances made byLangleben in automating the detection of lies through sophisticatedcomputer-based algorithms, the system still requires trained and skilledoperators.

Evoked Potentials

In neurophysiology, an evoked potential is an electrical potentialrecorded from a human or animal (or “biopotential”) following thepresentation of a stimulus, as distinct from spontaneous potentials suchas electroencephalograms or electromyograms. Evoked potential amplitudestend to be low, ranging from less than a microvolt to severalmicrovolts, compared to tens of microvolts for EEG, millivolts for EMG,and often close to a volt for EKG. To resolve these low-amplitudepotentials against the background of ongoing EEG, EKG, EMG and otherbiological signals and ambient noise, signal averaging is usuallyrequired. The signal is time-synchronized to the stimulus and since mostof the noise occurs randomly, the noise is averaged out.

Signals can be recorded from the cerebral cortex, brainstem, spinal cordand peripheral nerves. Usually the term “evoked potential” is reservedfor responses involving either recording from, or stimulation of,central nervous system structures. Thus evoked CMAP (compound motoraction potentials) or SNAP (sensory nerve action potentials) as used inNCV (nerve conduction studies) are generally not thought of as evokedpotentials, though they do meet the above definition.

Sensory evoked potentials (SEP or SSEP) are recorded from the centralnervous system following stimulation of sense organs (for example,visual evoked potentials elicited by a flashing light or changingpattern on a monitor; auditory evoked potentials by a click or tonestimulus presented through earphones) or by electrical stimulation of asensory or mixed nerve. They have been widely used in clinicaldiagnostic medicine since the 1970s, and also in intraoperativeneurophysiology monitoring (IONM), also known as surgicalneurophysiology. There are three kinds of evoked potentials inwidespread clinical use since the 1970s: auditory evoked potentials,usually recorded from the scalp but originating at brainstem level;visual evoked potentials, and somatosensory evoked potentials, which areelicited by electrical stimulation of peripheral nerves.

To measure evoked potentials electrodes need to be attached to variouspoints of on the scalp. Typically, the head is measured using astandardized EEG measurement technique to determine the optimallocations (each location corresponding to a type of EP that will bemeasured—e.g. the two locations on the back of the skull for the visualcortex, etc.), which are typically marked with magic marker. Each ofthese spots is rubbed with an oil-removing scrub to get rid of the skinoil. Then an electrode dipped in a liberal quantity of conductive gel isapplied to each location, and affixed with a strip of adhesive tape.

For visual evoked potentials (VEP), the subject is placed in front of acomputer screen, which shows a pattern of white and black squares like achessboard, and a red dot in the middle that the subject is supposed tofocus his/her eyes on with minimal movement. The procedure is done oneeye at a time, with the eye that is not being tested blocked off with aneye patch. During the actual procedure, these squares alternate (whiteones become black, black ones become white) at a rate of several times asecond. This produces responses in the visual cortex that are picked upby the electrodes. Since the computer controls the exact timing of thechanges of the square colors, and receives the electrical response inthe corresponding electrodes, it is able to determine precisely theamount of time it takes for the visual stimulus to reach the visualcortex. For the somatosensory evoked potentials (SEP), additionalelectrodes are applied, in the same manner as described earlier.

There are many things going on at once in the brain, so it is difficultto determine when the evoked potential from a particular stimulusarrives from just one stimulus. A common technique used to amplify thesignal is called ensemble averaging. The stimulus in each evokedpotential experiment is presented multiple times, and since other signalcomponents besides the evoked potential are not related to the signal,the computer can discriminate and amplify the one consistent peak orseries of peaks that are caused by the applied stimulus.

In the 1980's Towle, Heuer & Donchin demonstrated that a subject wouldproduce a positive signal peak approximately 300 msec after onset ofstimulus (P-300) in response to visual stimuli that consisted of twosets of photographs, one of generally known politicians, the other ofgenerally known movie stars. The subjects were instructed to count oneor the other category. Each time an image from the task-relevant classwas displayed, the subject produced the traditional P-300 response. Thisresearch confirmed what was well known since the mid 1960's, primarilythat P-300's are elicited by stimuli that provide information necessaryfor the performance of an explicit task assigned by theexperimenter—such as counting movie stars. It was also accepted thatP-300's would be not be present in the absence of such an explicit task.

In the late 1990's early 2000's Farwell et. al. hypothesized thatstimuli that are not explicitly task-relevant would still neverthelesselicit a P-300 if they are particularly significant to the subject dueto his/her past knowledge of the subject matter of the stimuli. Thistheory is based on the “Context Updating Model”. This model is based onthe idea that when a stimulus that is significant for the subject isprovided, he or she can be expected to take particular note of it, thusrevising his/her internal representation of the current environment andgenerating a P-300.

The Context Updating Model then provides an alternative means toascertaining concealed knowledge of a human subject. In contrast tofocusing on physiological changes elicited through an interrogativeprocess, a new technique for verifying knowledge can be created bymeasuring the response to known-relevant stimuli which serve as a proxyfor said knowledge. It is logical to assume that a human subject willproduce one kind of signal in response to images of people, places andthings for which he/she has knowledge and another kind of signal inresponse to images of people, places and things for which he/she has noknowledge.

There exist many methods using physiological metrics for the detectionof concealed information. The prior art is replete with examples ofvarious means and methods for ascertaining concealed knowledge of ahuman subject. Functional MRI has created a window to the mindpermitting scientists to observe the areas of the brain and the neuralpathways involved in lying. Many attempts, such as those disclosed byFarwell, utilize biopotentials including EEG in combination with aninterrogative process to determine if information provided by a humanuser is truthful, or if it is part of a subterfuge. Almost every exampleof “Truth Detection” in the prior art have two important limitationsthat are overcome by the present invention: (1) The process involves anauditory or visual interrogation of the human subject, and (2) Thesystem requires a skilled operator to administer the test and/or provideinterpretation of the results. This is the case with modern polygraphy,functional MRI and other physiologically-based technologies.

Many inventors have devised myriad of approaches attempting to provideinexpensive, minimally invasive, and rapid knowledge verificationsystems which could detect concealed knowledge (typically guiltyknowledge) of human users. However, none have succeeded in producing asystem that is practical and desirable for use in applications whereeither no direct interrogation is desired, or no trained operator isavailable to administer the examination or provide expert analysis ofthe results. Because of these and other significant limitations,commercially viable automated knowledge verification systems have notyet come to market.

The present invention overcomes all of the aforesaid limitations bycombining a system in which there is (1) no interrogation and (2) nosubjective human-expert analysis with robust hardware elements such as asimple headband, a rugged biopotential amplifier, a fast microprocessorand a user-friendly automated software interface. The present inventionutilizes an electrode-studded headband assembly that correctly positionsan array of disposable Ag/Ag—Cl electrodes in the desired anatomicalposition along the Z-axis of the head of the subject. To apply, theoperator places a dollop of conductive gel on the surface of eachelectrode, then lowers the band over and onto the subject's head andadjusts the tightness using Velcro straps. The leads from the headbandare subsequently connected to the biopotential amplifier which iselectrically attached to a computer such as a PC or laptop. In onepotential embodiment, with the subject seated before the video monitor(which provides the stimulating visual images), the operator starts thecontrol/analysis software, performs some simple calibration and testroutines and begins the examination. During the examination, thesoftware resident on the PC or laptop controls all aspects of thesystem's operation. At the end of the test, information relevant to theexamination is provided to the operator in both electronic and hard-copyformat.

This novel method of utilizing biopotentials coupled to the easy-to-useautomated control/analysis software overcome many significantlimitations of the prior art. Subjectivity and interpretation is notrequired with the present invention. Instead, the human-expert analysisof the prior art inventions is replaced by an objective softwareanalysis algorithm. The only training required by the present inventionis in its setup and operation—training that can be accomplished quicklyand inexpensively.

SUMMARY

It is an object of the present invention to overcome the problems,obstacles and deficiencies of the prior art.

It is also an object of the present invention to provide an improvedsystem for verifying concealed knowledge of a human subject utilizingbiopotentials and physiological metrics. It is further an object of thepresent invention to provide an improved system and method for enhancingthe training of human subjects.

Accordingly, one embodiment of the present invention is directed to asystem and method for verifying human knowledge by means of measuringsubject's physiological responses to input stimuli comprising: (a) astimuli exposure system that exposes the subject to input stimuli; (b) asensor system that monitors and records subject's physiologicalresponses before, during, and after input stimulus is presented tosubject; (c) a computer system comprising processing and algorithmicelements that determines if subject's physiological responses indicatesubject possesses knowledge of interest; and (d) a reporting system thatpresents a report to the operator which indicates: (i) input stimuliexposed to subject, (ii) physiological responses of the subject before,during, and after each input stimulus is exposed to subject, and (iii)determination of the computer system of whether subject possessesknowledge of interest.

The stimuli exposure system of this first embodiment includes Probe,Relevant and Gallery data; and a visual display comprising an LCD videomonitor. A Protocol Creation Algorithm presents the probe, relevant andgallery visual stimuli in a weighted pseudo-random sequence. The Probeimage data are not generally known to human subjects but relevant to theknowledge to be verified. The relevant image data are generally known tohuman subjects but not relevant to the knowledge to be verified. Thegallery image data are not generally known to human subjects and notrelevant to the knowledge to be verified.

Although exclusively visual stimuli are described in the firstembodiment, alternative stimuli such as auditory, tactile, and olfactorystimuli may also be presented in other embodiments. These may bepresented independently or in combination. The stimuli exposure systempresents auditory stimuli through recorded audio playback. Tactile andolfactory stimuli are presented through physical samples.

The sensor system of this first embodiment includes a removablephysiological data amplifier connected to a human subject via disposableAg/Ag—Cl electrodes and an analog-to-digital (A/D) converter to digitizephysiological data for subsequent storage on the computer system.Although this first embodiment reads specifically biopotential signals,additional physiological signals may also be monitored. These includefunctional magnetic resonance images or positron emission tomography.The data gathering would require the use of a Functional MagneticResonance Imaging (fMRI) machine or Positron Emission Tomography (PET)scanner.

The computer system of this first embodiment includes data analysissoftware. Analysis software discriminates subject's event-relatedresponse from exogenous stimuli by means of neural network analysis,parametric analysis, statistical analysis, pattern matching, or waveletprocessing to provide an output of verification or non-verification ofknowledge of interest. The reporting system presents the results of thedetermination of the data analysis software.

When an image is recognized, the user would be expected to produce aP-300 response indicative of said recognition. A predetermined number ofinstantiations of the Probe, Relevant and Gallery images would beensemble averaged to ameliorate the contributions of artifacts. Byexamining the ratio of recognized-to-nonrecognized Probe and Galleryimages, an objective metric for relevant knowledge verification could berealized. Statistical evaluation of the false-positives (incorrectlyrecognizing a Gallery image as a Probe image) and false-negatives(failing to correctly recognize a Probe image) for a specific subjectwould provide a determination with respect to subject's relevantknowledge. Relevant images provide benchmark measurements for recognizednon-relevant images.

A second embodiment of the present invention is directed to a system andmethod for enhancing the training of human subjects in which subject'ssuccessful assimilation of information provided at a prior trainingevent is verified by means of measuring subject's physiologicalresponses to input stimuli. This second embodiment is comprised of themethod and apparatus of the first embodiment in addition to a methodwherein the results of the verification of subject's successfulassimilation of information provided at a prior training event are usedto determine if subject should repeat the training event. In addition,these results can be used to improve future training events.

With respect to this method, subject would complete a training exerciseand subsequently be tested to verify relevant knowledge and determineretention. This test could be conducted using objective metrics. Anelectronic hand-operated switch could be optionally provided to the userto depress each time an image is recognized. Alternatively, subjectcould be asked to perform an abstract cognitive task such as avisualization task or counting task when an image is recognized. Theseresults can further be utilized in modifying the training curriculum ortraining technique to optimize the exercise. If a particular set ofinformation was not being taught well, the results of the test wouldshow a consistent lack of knowledge in that area. The teaching methodsfor that area could then be changed in future training.

Other objects and advantages will be readily apparent to those ofordinary skill in the art upon viewing the drawings and reading thedetailed description hereinafter.

DRAWINGS Figures

FIG. 1 shows a block diagram of the software architecture and hardwareinterfaces for the system design.

FIG. 2 shows in flow diagram a representation of the general processingsteps of the concealed knowledge verification system.

FIG. 3 shows in exploded view a representation of the headgear for thepresent invention.

FIG. 4 shows in functional block diagram an exemplar screening module ofthe present invention.

FIG. 5 shows in functional block diagram a dynamically adaptableknowledge tree utilized with the present invention.

FIG. 6 shows in image form exemplars of the Probe, Relevant and Galleryimages utilized with the present invention.

FIG. 7 shows an exemplar event data tracing of the present invention.

FIG. 8 shows in graphical form a representation of a parametriccorrelation plotting analysis technique of the present invention.

FIG. 9 shows in functional block diagram a representation of a neuralnetwork of the concealed knowledge verification system.

FIG. 10 shows in flow diagram a representation of the general processingsteps of another aspect of the present invention for monitoring andenhancing training.

DRAWINGS - Reference Numerals 100 computer-based device 200 generalprocessing steps 300 headgear with biopotential sensors 400 screeningmodule 500 knowledge tree 600 gallery data 700 event data tracing 800combined correlations plotting analysis 900 neural network 1000 flowchart

DETAILED DESCRIPTION

Although those of ordinary skill in the art will readily recognize manyalternative embodiments, especially in light of the illustrationsprovided herein, this detailed description is of the preferredembodiment of the present invention, a system and method for knowledgeverification utilizing biopotentials and physiologic metrics.

In general, as shown in FIG. 1, a system and method for the verificationof concealed knowledge of the present invention is referred to by thenumeral 100 and generally comprises a computer-based device 110,software for performing the steps of the invention 120, operator controlinterface 130, and human subject interface 140.

Referring now specifically to FIG. 1, a system and method for knowledgeverification utilizing biopotentials and physiologic metrics includes acomputer 110 well known in the art and commercially available under suchtrademarks as IBM®, Compaq®, and Dell® having a central processor (CP)111 that is well known in the art and commercially available under suchtrademarks as Intel® 486, Pentium®, and Motorola 68000, conventionalnon-volatile Random Access Memory (RAM) 112, conventional Read OnlyMemory (ROM) 113, and disk storage device(s) 114.

Computer 110 can be configured as a standard Personal Computer (PC) orcan be implemented as a custom single-board computer utilizing anembedded operating system such as is sold commercially under thetrademark Windows NT®. Computer 110 is operably associated withcommunications channel 115, which can be a conventional RS-232, USB oranother equivalent bi-directional communications port. Communicationschannel 115 has associated therewith an Analog-to-Digital converter 116,which can be one of myriad devices that are known to anyone of ordinaryskill in the art. The Analog-to-Digital converter 116 and communicationschannel 115 are responsible for converting the analog human biopotentialsignals into a digital representation that can be subsequently processedby computer 110. Computer 110 is further operably associated with diskstorage device(s) 114 comprising a file system utilized in storing theimages, protocols and human biopotential data 180. Computer 110 presentssequential images based on a testing protocol, described in detailherein below, to a human subject 141 via a video monitor 142electrically associated with computer 110 which can be of an LCD typewell known to anyone of ordinary skill in the art.

Human subject 141 has removably associated therewith Ag/Ag—Cl electrodearray headgear 143, the particulars of which are described hereinafter.Two biopotential amplifiers 144 and an analog communications channel 145that transmits three analog data types (Electroencephalogram (EEG),Electrooculogram (EOG), and Electromyogram (EMG)) is electricallyassociated with the Analog-to-Digital converter 116. Collectively, theseelements (141 through 145) are housed in a subject isolation booth 146designed to minimize artifacts caused by exogenous distractions andextraneous electrical noise. Graphical User Interface (GUI) 130 is alsoelectrically associated with computer 110 and provides the controlinterface for the operator 131. GUI 130 would generally include a mouse,keyboard and monitor (not shown) for interacting with computer 110.Computer 110 is further electrically associated with an optionalhand-operated switch 117 which can be located within the subjectisolation booth 146 and used for certain protocols describedhereinafter.

Computer 110 has programmably associated therewith software 120, whichcomprises a Wave Analysis Program 150, Wave Analysis Access Dynamic LinkLibrary (DLL) 170, and Data Processing and Control Program 161, theparticulars of which are further described hereinafter.

Wave Analysis Program 150 is comprised of Image Sequencing Logicalgorithm 155, Protocol Creation Algorithm 154, Ensemble Averager 153,and parametric or non-parametric analysis algorithm(s) 151.

Image Sequencing Logic algorithm 155 is programmably associated withLiquid Crystal Display (LCD) video monitor 14, Wave Analysis Access DLL170 and disk storage device(s) 114. Probe, Relevant, and Gallery images,described in detail herein below, are stored in the file system on diskstorage device 114 and displayed to human subject 141 in a predeterminedstatistically weighted, pseudo-random sequence generated by ProtocolCreation Algorithm 154. Image sequencing logic algorithm 155 is furtherprogrammably associated with Wave Analysis Access DLL 170 providingstatus updates via data conduit 156. This status data provides controland sequencing information to Data Processing and Control Program 161via communications channels (171 through 178 inclusive) permittingtime-synchronized data collection.

Wave Analysis Access DLL 170 is programmably associated with DataProcessing and Control Program 161 through data conduits 171 through178. These data conduits are comprised of Start Session 171, StartRecording 172, Image Displayed 173, Image Blanked 174, Stop Recording175, Stop Session 176, Ready 177 and Start 178. In response thereto tosignals sent via data conduits 171 through 178, Data Processing andControl Program 161 provides for the control and recordation of thehuman biopotential data 180 within disk storage device(s) 114.

Data Processing and Control Program 161 is electrically associated withcommunications port 115 and analog-to-digital converter 116, and isprimarily responsible for controlling the data collection of humanbiopotential data 180 and processing and storing said data. DataProcession and Control Program 161 has programmably associated therewitha Lock-in Amplifier (LIS) algorithm, which is well known to anyone ofordinary skill in the art, and for the preferred embodiment, functionsas a very-high Q filter. LIS algorithm computes a time-history of thepower spectrum for a single predetermined frequency in near real-time.Multiple instantiations of LIS algorithm can be run concomitantly on PC110, each with a unique pre-determined frequency. Since brain functionproduces signals that can be grouped into discrete bands of frequencies,LIS algorithm provides a way to discern information about what the brainis doing at any given point in time.

Including the raw EEG, EOG, and EMG data transmitted via communicationchannel 145 and data generated by LIS algorithm, Data Processing andControl Program 161 captures and records 26 simultaneous channels (13for each bioamplifier 144) of data on disk storage device(s) 114.Finally, hand-operated switch 117 is electrically and programmablyassociated with Data Processing and Control Program 161 to provide anoptional input from subject 141 that can be used for certain concealedknowledge verification protocols or for testing/calibration of system100.

As shown in FIG. 2, general processing steps 200 appropriate forimplementation of the present invention include: preparing subject 201,calibrating bioamplifier 202, generating protocol for pass 1 203, andrunning initial screening module 204. At that point, the data is stored205, the pass 2 protocols are generated 206, and subcategory paths arethen run using decision trees 207. This data is stored for analysis 208,and the final analysis is completed 209. Based on this analysis, thesubcategory paths are either rerun or human subject disconnects fromheadgear 210, and written report 211 is generated.

FIG. 3 is a drawing of headgear 300 with biopotential sensors 304,305,306, 307, 308, and 309. Strap 301 mounts sensors to the top of thehead, and strap 302 mounts them horizontally around the head from thefront to the rear where it is attached at fastener 303. Transmissionwires 311 transmit the biopotential signals to the biopotentialamplifier 312.

FIG. 4 is a drawing of screening module 400 for this embodiment,indicating person of interest 401 and examples of categories of interest402.

FIG. 5 is an example of knowledge tree 500, which might be used in asubcategory routine. The highest level of the tree, weapons/contraband501, represents the broadest level of indication, while specificammunition 511, represents the most specific level of indication. Ifweapons/contraband 501 evokes a response, then artillery projectiles502, handheld firearms 503, mortars-launchers 504, and mines/ExplosivelyFormed Penetrators (EFPs) 505 will all be tested. If none of thesestimuli evoke a response, then the tree will terminate with theweapons/contraband indication. However, if any of 502-505 evokes aresponse, then the routine will continue to work its way through RocketPropelled Grenades (RPGs) 506, small arms 507, grenades 508, close-inpictures 509, disassembled parts 510, and specific ammunition 511 aslong as at least one category on each level evokes a positive response.

FIG. 6 is an example of possible sets of Probe, Relevant, and Gallerydata. 610 represents normal individuals with whom human subject wouldgenerally not be familiar. 620 represents political figures of whomhuman subject would have knowledge. 630 represents common buildingsabout which hum subject would generally not be familiar. 640 representsfamous or iconic buildings about which human subject would haveknowledge. 650 represents automobiles and perfume brands that humansubject would not normally have encountered before. 660 representsfamous or common automobiles and perfume brands that human subject wouldhave knowledge of.

FIG. 7 is an example of event data tracing 700 from the presentembodiment. 701 is the Sheep test. 710 indicates the return to normalpattern after the stimulus. 720 and 730 show the reaction to thestimulus without recognition of the content of the stimulus. 702 is theWolf test. 760 shows the P300 recognition response to the stimulus, and740 and 750 show the signal returning to a normal pattern.

FIG. 8 includes diagrams of combined correlations plotting analysis 800for the Sheep 810 and Wolf 820 tests of FIG. 7.

As shown in FIG. 9, the neural network 900 includes at least one layerof trained neuron-like units, and preferably at least three layers. Theneural network 900 includes input layer 970, hidden layer 972, andoutput layer 974. Input layer 970, hidden layer 972, and output layer974 include a plurality of trained neuron-like units 976, 978 and 980,respectively.

Neuron-like units 976 can be in the form of software or hardware. Theneuron-like units 976 of the input layer 970 include a receiving channelfor receiving human biopotential data, wherein the receiving channelincludes modulator 975 for modulating the signal.

The neuron-like units 978 of the hidden layer 972 are individuallyreceptively connected to each of the units 976 of the input layer 970.Each connection includes a predetermined modulator 977 for modulatingeach connection between the input layer 970 and the hidden layer 972.The neuron-like units 980 of the output layer 974 are individuallyreceptively connected to neuron-like units 978 of hidden layer 972. Eachconnection includes predetermined modulator 979 for modulating eachconnection between hidden layer 972 and output layer 974. Each unit 980of said output layer 974 includes an outgoing channel for transmittingthe output signal.

Each neuron-like unit 976, 978, and 980 includes dendrite-like unit 960,and preferably several, for receiving incoming signals. Eachdendrite-like unit 960 includes modulator 975, 977, and 979, whichmodulates the amount of weight that is to be given to the particularcharacteristic sensed as described below. In dendrite-like unit 960,modulator 975, 977, and 979 modulate the incoming signal andsubsequently transmit a modified signal 962. For software, dendrite-likeunit 960 comprises an input variable X and a weight value W wherein theconnection strength is modified by multiplying the variables together.For hardware, dendrite-like unit 960 can be a wire, optical orelectrical transducer having a chemically, optically or electricallymodified resistor therein.

Neuron-like units 976, 978, and 980 include soma-like unit 963, whichhas a threshold barrier defined therein for the particularcharacteristic sensed. When soma-like unit 963 receives modified signal962, this signal must overcome the threshold barrier whereupon aresulting signal is formed. Soma-like unit 963 combines resultingsignals 962 and equates the combination to output signal 964 indicativeof the response to the collective inputs.

For software, soma-like unit 963 is represented by the sum Σ=G_(a)X_(a)W_(a)−β, where β is the threshold barrier. This sum is employed ina Nonlinear Transfer Function (NTF) as defined below. For hardware,soma-like unit 963 includes a wire having a resistor; the wiresterminating in a common point that feeds into an operational amplifierhaving a nonlinear component which can be a semiconductor, diode, ortransistor.

Neuron-like unit 976, 978, and 980 include axon-like unit 965 throughwhich the output signal travels, and also includes at least onebouton-like unit 966, and preferably several, which receive the outputsignal from axon-like unit 965. Bouton/dendrite linkages connect inputlayer 970 to hidden layer 972 and hidden layer 972 to output layer 974.For software, axon-like unit 965 is a variable which is set equal to thevalue obtained through the NTF and bouton-like unit 966 is a functionwhich assigns such value to dendrite-like unit 960 of the adjacentlayer. For hardware, axon-like unit 965 and bouton-like unit 966 can bea wire, an optical or electrical transmitter.

Modulators 975, 977, and 979, which interconnect each of the layers ofneurons 970, 972, and 974 to their respective inputs, determine theclassification paradigm to be employed by neural network 900. Humanbiopotential data are provided as inputs to the neural network and theneural network subsequently characterizes and generates an output signalin response thereto which is one of a categorization of the humanbiopotential data.

It is not exactly understood what weight is to be given tocharacteristics that are modified by the modulators of the neuralnetwork, as these modulators are derived through a training processdefined below.

The training process is the initial process that the neural network mustundergo in order to obtain and assign appropriate weight values for eachmodulator. Initially, modulators 975, 977, and 979 and the thresholdbarrier are all assigned small random, non-zero values. The modulatorscan each be assigned the same value, but the neural network's learningrate is best maximized if random values are chosen. Human biopotentialdata 180 are fed in parallel into the dendrite-like units of the inputlayer (one dendrite connecting to each data point of the humanbiopotential data 180) and the output observed.

The Nonlinear Transfer Function (NTF) employs a gain factor g in thefollowing equation to arrive at the output:

NTF=1/[1+e ^(−g)]

For example, in order to determine the amount weight to be given to eachmodulator for any given human facial image, the NTF is employed asfollows:

If the NTF approaches 1, the soma-like unit produces an output signalindicating a strong response. If the NTF approaches 0, the soma-likeunit produces an output signal indicating a weak response. If the outputsignal clearly conflicts with the known empirical output signal, anerror occurs. The weight values of each modulator are adjusted using thefollowing formulas so that the input data produces the desired empiricaloutput signal.

For the output layer:

W* _(kol) =W _(kol) +GE _(k) Z _(kos)

-   -   W*_(kol)=new weight value for neuron-like unit k of the outer        layer    -   W_(kol)=current weight value for neuron-like unit k of the outer        layer    -   G=gain factor    -   Z_(kos)=actual output signal of neuron-like unit k of output        layer    -   D_(kos)=desired output signal of neuron-like unit k of output        layer

E _(k) =Z _(kos)(1−Z _(kos))(D _(kos) −Z _(kos)),

-   -   (this is an error term corresponding to neuron-like unit k of        outer layer).

For the hidden layer:

W* _(jhl) =W _(jhl) +GE _(j) Y _(jos)

-   -   W*_(jhl)=new weight value for neuron-like unit j of the hidden        layer.    -   W_(jhl)=current weight value for neuron-like unit j of the        hidden layer.    -   G=gain factor    -   Y_(jos)=actual output signal of neuron-like unit j of hidden        layer.

E _(j) =Y _(jos)(1−Y _(jos))E _(k)(E _(k) *W _(kol)),

-   -   (this is an error term corresponding to neuron-like unit j of        hidden layer over all k units).

For the input layer:

W* _(iil) =W _(iil) +GE _(i) X _(ios)

-   -   W*_(iil)=new weight value for neuron-like unit I of input layer.    -   W_(iil)=current weight value for neuron-like unit I of input        layer.    -   G=gain factor    -   X_(ios)=actual output signal of neuron-like unit I of input        layer.

E _(i) =X _(ios)(1−X _(ios))E _(j)(E _(j) *W _(jhl)),

-   -   (this is an error term corresponding to neuron-like unit of        input layer over all j units).

The training process consists of entering new (or the same) exemplardata into neural network 900 and observing the output signal withrespect to a known empirical output signal. If the output is in errorwith what the known empirical output signal should be, the weights areadjusted in the manner described above. This iterative process isrepeated until the output signals are substantially in accordance withthe desired (empirical) output signal, and then the weight of themodulators are fixed. Upon fixing the weights of the modulators, theneural network is then trained and can make generalizations about humanbiopotential input data that is new to the neural network.

The description provided for neural network 900 as utilized in thepresent invention is but one technique by which a neural networkalgorithm can be employed. It will be readily apparent to those who areof ordinary skill in the art that numerous neural network paradigmsincluding multiple (sub-optimized) networks, as well as numeroustraining techniques, can be employed to obtain equivalent results to themethod as described herein above. In addition, myriad techniques forpreprocessing said human biopotential input data can be employed tobetter prime the data for presentation to a neural network algorithm.These techniques can help create an input signal that isscale-normalized and translationally invariant and subsequently reduceerror contributions due to the sensitivity of neural networks to theseparameters.

FIG. 10 contains flow chart 1000 for a system and method for monitoringand enhancing student training programs. First the subject is prepared1001, the bioamplifier is calibrated 1002, and a protocol for evaluatingthe first area of training knowledge is generated 1003. Then knowledgeverification run module 1004 executes and the results of the data arestored 1005. At step 1006, the additional knowledge verification runmodules may execute if there are additional knowledge areas to test.After there are no further knowledge areas to test, the results of thetest are evaluated on a pass/fail basis at 1007. If the subject did notpass, then he or she will have to repeat the sequence of run modules ata later time at 1008. If the subject did pass, then the final analysisis completed at 1009 and he or she is disconnected at 1010. Lastly, awritten report is generated based on the results of final analysis 1011.

The above described embodiments are set forth by way of example and arenot for the purpose of limiting the scope of the present invention. Itwill be readily apparent to those or ordinary skill in the art thatobvious modifications, derivations and variations can be made to theembodiments without departing from the scope of the invention. Forexample, the automated human biopotential analysis software describedherein above as either non-parametric analysis algorithm, such as neuralnetwork, or parametric analysis algorithm could also be one of astatistical based system, template or pattern matching, or evenrudimentary wavelet processing techniques whereby the characteristics ofthe biopotential signals are analyzed. Similarly, the data processingand control program described in detail above as utilizing lock-inamplifier, could be one of many other algorithms well known to anyone ofordinary skill in the art.

1. A knowledge verification method, comprising presenting, on a monitoroperatively connected to a computer, probe, relevant, and gallery datato a subject in a statistically weighted, pseudo-random sequence througha control program operating on the computer; recording on the computer,a time history of the presentation of the probe, relevant, and gallerydata; recording on the computer, while the subject is experiencing thepresentation of the probe, relevant, and gallery data, electricalactivity in the subject's brain that is measured by one or moreelectrodes operatively connected to the computer; determining, throughdata analysis software operating on the computer, whether the electricalactivity contains one or more p-300 responses; and determining, throughthe data analysis software operating on the computer, whether any of thep-300 responses are correlated with the subject experiencing the probedata by comparing the occurrence of any p-300 responses to the timehistory of the probe, relevant, and gallery data presented to thesubject.
 2. The method of claim 1, wherein the presenting probe,relevant, and gallery data to the subject takes place while the subjectis in an isolation booth that substantially reduces exogenousdistractions and extraneous electrical noise.
 3. The method of claim 1,wherein the presenting probe, relevant, and gallery data proceedsaccording to a knowledge tree where the broadest category of probe datais presented first and more specific categories of the same generalcategory of probe data are only presented if broader categories evoke ap-300 response.
 4. The method of claim 1, wherein the probe, relevant,and gallery data make up one or more sets of stimulus files, and thepresenting of probe, relevant, and gallery data further comprises:presenting the sets serially from broad divisions of information toinformation of progressively greater specificity to discover selectedknowledge, and presenting subsequent sets based on whether the stimulusfiles indicate recognition of a stimulus.
 5. The method of claim 1,wherein the data analysis software implements a neural network thatemploys at least three layers of neuron-like units.
 6. The method ofclaim 1, wherein the data analysis software implements a neural networkthat employs ensemble averaging.
 7. The method of claim 1, wherein thedata analysis software implements a neural network that employs one ormore weighted modulators between each layer of neuron-like units.
 8. Aknowledge verification system, comprising one or more electrodes; acomputer operatively connected to the one or more electrodes and havinginstructions encoded thereon for: presenting probe, relevant, andgallery data to a subject in a statistically weighted, pseudo-randomsequence; recording a time history of the presentation of the probe,relevant, and gallery data; recording, while the subject is experiencingthe presentation of the probe, relevant, and gallery data, electricalactivity in the subject's brain that is measured by the one or moreelectrodes; determining whether the electrical activity contains one ormore p-300 responses; and determining whether any p-300 responses arecorrelated with the subject experiencing the probe data by comparing theoccurrence of any p-300 responses to the time history of the probe,relevant, and gallery data presented to the subject.
 9. The system ofclaim 8, further comprising an isolation booth that substantiallyreduces exogenous distractions and extraneous electrical noise.
 10. Thesystem of claim 8, wherein presenting probe, relevant, and gallery dataproceeds according to a knowledge tree where the broadest category ofprobe data is presented first and more specific categories of the samegeneral category of probe data are only presented if broader categoriesevoke a p-300 response.
 11. The system of claim 8, wherein the probe,relevant, and gallery data make up one or more sets of stimulus files,and the presenting of probe, relevant, and gallery data furthercomprises: presenting the sets serially from broad divisions ofinformation to information of progressively greater specificity todiscover selected knowledge, and presenting subsequent sets based onwhether the stimulus files indicate recognition of a stimulus.
 12. Thesystem of claim 8, wherein determining whether the electrical activitycontains one or more p-300 responses is achieved by employing a neuralnetwork having at least three layers of neuron-like units.
 13. Thesystem of claim 8, wherein determining whether the electrical activitycontains one or more p-300 responses is achieved by employing a neuralnetwork employing ensemble averaging.
 14. The system of claim 8, whereindetermining whether the electrical activity contains one or more p-300responses is achieved by employing a neural network having neuron-likeunits with one or more weighted modulators.
 15. A knowledge verificationsystem, comprising one or more electrodes; a computer operativelyconnected to the one or more electrodes and having instructions encodedthereon for: presenting probe, relevant, and gallery data to a subjectin a statistically weighted, pseudo-random sequence; recording a timehistory of the presentation of the probe, relevant, and gallery data;recording, while the subject is experiencing the presentation of theprobe, relevant, and gallery data, electrical activity in the subject'sbrain that is measured by the one or more electrodes; determiningwhether the electrical activity contains one or more event-relatedpotentials; and determining whether any event-related potentials arecorrelated with the subject experiencing the probe data by comparing theoccurrence of any event-related potentials to the time history of theprobe, relevant, and gallery data presented to the subject.
 16. Thesystem of claim 15, wherein presenting probe, relevant, and gallery dataproceeds according to a knowledge tree where the broadest category ofprobe data is presented first and more specific categories of the samegeneral category of probe data are only presented if broader categoriesevoke an event-related potential.
 17. The system of claim 15, whereinthe probe, relevant, and gallery data make up one or more sets ofstimulus files, and the presenting of probe, relevant, and gallery datafurther comprises: presenting the sets serially from broad divisions ofinformation to information of progressively greater specificity todiscover selected knowledge, and presenting subsequent sets based onwhether the stimulus files indicate recognition of a stimulus.
 18. Aknowledge verification system, comprising one or more electrodes; acomputer operatively connected to the one or more electrodes and havinginstructions encoded thereon for: presenting probe, relevant, andgallery data to a subject in a statistically weighted, pseudo-randomsequence; recording a time history of the presentation of the probe,relevant, and gallery data; recording, while the subject is experiencingthe presentation of the probe, relevant, and gallery data, electricalactivity in the subject's brain that is measured by the one or moreelectrodes; determining whether the electrical activity contains one ormore event-related potentials having a latency of approximately 250 msto 900 ms; and determining whether any of the event-related potentialshaving a latency of approximately 250 ms to 900 ms are correlated withthe subject experiencing the probe data by comparing the occurrence ofthe event-related potentials having a latency of approximately 250 ms to900 ms to the time history of the probe, relevant, and gallery datapresented to the subject.
 19. The system of claim 18, wherein presentingprobe, relevant, and gallery data proceeds according to a knowledge treewhere the broadest category of probe data is presented first and morespecific categories of the same general category of probe data are onlypresented if broader categories evoke an event-related potential havinga latency of approximately 250 ms to 900 ms.
 20. The system of claim 18,wherein the probe, relevant, and gallery data make up one or more setsof stimulus files, and the presenting of probe, relevant, and gallerydata further comprises: presenting the sets serially from broaddivisions of information to information of progressively greaterspecificity to discover selected knowledge, and presenting subsequentsets based on whether the stimulus files indicate recognition of astimulus.